18 research outputs found
Deep Latent State Space Models for Time-Series Generation
Methods based on ordinary differential equations (ODEs) are widely used to
build generative models of time-series. In addition to high computational
overhead due to explicitly computing hidden states recurrence, existing
ODE-based models fall short in learning sequence data with sharp transitions -
common in many real-world systems - due to numerical challenges during
optimization. In this work, we propose LS4, a generative model for sequences
with latent variables evolving according to a state space ODE to increase
modeling capacity. Inspired by recent deep state space models (S4), we achieve
speedups by leveraging a convolutional representation of LS4 which bypasses the
explicit evaluation of hidden states. We show that LS4 significantly
outperforms previous continuous-time generative models in terms of marginal
distribution, classification, and prediction scores on real-world datasets in
the Monash Forecasting Repository, and is capable of modeling highly stochastic
data with sharp temporal transitions. LS4 sets state-of-the-art for
continuous-time latent generative models, with significant improvement of mean
squared error and tighter variational lower bounds on irregularly-sampled
datasets, while also being x100 faster than other baselines on long sequences
Learning Efficient Surrogate Dynamic Models with Graph Spline Networks
While complex simulations of physical systems have been widely used in
engineering and scientific computing, lowering their often prohibitive
computational requirements has only recently been tackled by deep learning
approaches. In this paper, we present GraphSplineNets, a novel deep-learning
method to speed up the forecasting of physical systems by reducing the grid
size and number of iteration steps of deep surrogate models. Our method uses
two differentiable orthogonal spline collocation methods to efficiently predict
response at any location in time and space. Additionally, we introduce an
adaptive collocation strategy in space to prioritize sampling from the most
important regions. GraphSplineNets improve the accuracy-speedup tradeoff in
forecasting various dynamical systems with increasing complexity, including the
heat equation, damped wave propagation, Navier-Stokes equations, and real-world
ocean currents in both regular and irregular domains.Comment: Published as a conference paper in NeurIPS 202
What if We Enrich day-ahead Solar Irradiance Time Series Forecasting with Spatio-Temporal Context?
Solar power harbors immense potential in mitigating climate change by
substantially reducing CO emissions. Nonetheless, the inherent
variability of solar irradiance poses a significant challenge for seamlessly
integrating solar power into the electrical grid. While the majority of prior
research has centered on employing purely time series-based methodologies for
solar forecasting, only a limited number of studies have taken into account
factors such as cloud cover or the surrounding physical context. In this paper,
we put forth a deep learning architecture designed to harness spatio-temporal
context using satellite data, to attain highly accurate \textit{day-ahead}
time-series forecasting for any given station, with a particular emphasis on
forecasting Global Horizontal Irradiance (GHI). We also suggest a methodology
to extract a distribution for each time step prediction, which can serve as a
very valuable measure of uncertainty attached to the forecast. When evaluating
models, we propose a testing scheme in which we separate particularly difficult
examples from easy ones, in order to capture the model performances in crucial
situations, which in the case of this study are the days suffering from varying
cloudy conditions. Furthermore, we present a new multi-modal dataset gathering
satellite imagery over a large zone and time series for solar irradiance and
other related physical variables from multiple geographically diverse solar
stations. Our approach exhibits robust performance in solar irradiance
forecasting, including zero-shot generalization tests at unobserved solar
stations, and holds great promise in promoting the effective integration of
solar power into the grid
Port-Hamiltonian Approach to Neural Network Training
Neural networks are discrete entities: subdivided into discrete layers and
parametrized by weights which are iteratively optimized via difference
equations. Recent work proposes networks with layer outputs which are no longer
quantized but are solutions of an ordinary differential equation (ODE);
however, these networks are still optimized via discrete methods (e.g. gradient
descent). In this paper, we explore a different direction: namely, we propose a
novel framework for learning in which the parameters themselves are solutions
of ODEs. By viewing the optimization process as the evolution of a
port-Hamiltonian system, we can ensure convergence to a minimum of the
objective function. Numerical experiments have been performed to show the
validity and effectiveness of the proposed methods.Comment: To appear in the Proceedings of the 58th IEEE Conference on Decision
and Control (CDC 2019). The first two authors contributed equally to the wor